Autonomic Workflow Management System for Grid Computing

As many of the large-scale scientific applications executed on Grids are expressed as complex scientific workflows, workflow management has emerged as one of the most important Grid services in past few years. Scientific workflows can be defined as the aggregation of Grid application services, which are executed on distributed Grid resources in a well defined order to satisfy the specific requirements of users. A workflow management system is generally employed to define, manage and execute these workflows in world-wide Grid environment. However, the increasing scale complexity, heterogeneity and dynamism of Grid environment that includes networks, resources and applications have made such workflow management systems brittle, unmanageable and insecure. Autonomic computing provides a holistic approach for the design and development of systems/applications that can adapt themselves to meet the requirements of performance, fault tolerance, reliability, security, etc., without manual intervention. Therefore, we aim to develop algorithms (i.e. workflow scheduling strategy, resource coordination scheme) that aid the workflow management systems to incorporate the properties of autonomic computing and exhibit the ability to reconfigure itself to the changes in Grid environment, discover, diagnose and react to the disruptions of workflow execution as well as monitor and optimize its performance automatically. To this end, we leverage a distributed hash table based Peer-to-Peer (P2P) overlay to build a logical structure for the resource organization. The scalable and self-organizing nature of the P2P overlay provides a seamless framework for Grid resources to automatically self-configure themselves in the event of unavoidable situations, such as resource join, leave or failure. Further, we develop a reputation based dependable scheduling algorithm that improves the reliability of workflow execution through proactive resource provisioning by taking into account the prior performance and behaviour of Grid resources. Thus, this thesis makes several contributions towards improving the state-of-the-art of autonomic workflow management systems for Grid computing environment and particularly, towards advancing the area of Grid workflow scheduling. The major contributions are: (i) proposed a taxonomy of autonomic application management for Grid computing and surveyed existing Grid systems; (ii) developed a dynamic critical path based workflow scheduling algorithm that can adapt to changing Grid environment; (iii) designed an architecture for autonomic workflow management system in Grids according to the requirements identified in proposed taxonomy; (iv) devised a decentralized and cooperative workflow scheduling algorithm, utilizing a self-configuring P2P overlay structure with regards to resource discovery, coordination and overall system decentralization; and (v) leveraging the proposed autonomic workflow management architecture, developed a reputation-based dependable workflow scheduling technique to enable self-healing behavior in Grid workflow management system. This is to certify that (i) the thesis comprises only my original work, (ii) due acknowledgement has been made in the text to all other material used, (iii) the thesis is less than 100,000 words in length, exclusive of table, maps, bibliographies, appendices and footnotes.